Instruments of cognition: Use of citations and Web links in online teaching materials

Use of citations and web links embedded in online teaching materials was studied for an undergraduate course. The undergraduate students enrolled in Geographic Information Science for Geography and Regional Development used web links more often than citations, but clearly did not see them as key to enhancing learning. Current conventions for citing and linking tend to make citations and links invisible. There is some evidence that citations and web links categorized and highlighted in terms of their importance and function to be served may help student learning in interdisciplinary domains. Instruments of Cognition 3 Introduction Citations serve a variety of socio-cognitive functions; they signify credit and communicate the existence of related literature. Merton summarized the role of citations in maintaining the character and nature of science in this statement: “Citations provide pellets of the peer recognition that is central to the normative and reward system of science.” (Merton, 2000, p. 438). Thus, citations serve two different functions, instrumental cognitive and symbolic institutional functions. The cognitive functions include leading readers to assess for themselves the validity of claims made in the citing paper by following the leads given to other documents on which these claims rest or which are related, draw upon relevant sources and in turn be directed to other sources not mentioned in the citing publication. The symbolic functions maintain the character of scientific intellectual property. Merton (op cit.) further notes that the latter aspect has been studied most. “To this day, however, no systematic empirical research has determined the extent to which this potential cognitive function has actually been realized in the various scientific disciplines.” We report a modest effort to systematically explore the use of citations by learners, in undergraduate Geographic Information Science (GIS) course materials, in order to elucidate their role in learning a new area of study. Reasons for using citations in research are well documented. Cronin (1984) summarizes two theories of citing. The normative theory acknowledges the intellectual uses of others’ work and thus citations represent scientists’ influence and impact; the microsociological school doubts the normative assumptions and explores the roles of citations and cited documents, the motivations for citation practice, and the functions and quality of citations. Few studies investigate citations in the scholarly products of Instruments of Cognition 4 teaching and learning. The normative approach has long considered citations to peer– reviewed research articles as indicators of scholarly prominence, but anecdotal evidence suggests that these may no longer be the only useful measure. Citations and hyperlinks to academic products such as web pages in support of teaching can be valuable in increasing one’s reputation and prestige over a larger sphere of influence than citation indexes have normally captured and documented (Cronin et al, 1998; Vaughan & Shaw, 2003). It is possible to improve our understanding of citation use behaviors. For example, extending investigations beyond the citer’s motivation or behavior in order to look at how citations are used in information seeking (Wang & Soergel, 1998; Wang & White, 1999) or while reading would provide a more complete view. Studying citation use in the context of formal teaching and learning activities is important because footnote chasing and citation searching have been shown to be specific uses of references in differing learning contexts such as writing the academic term paper and continuing professional education. Often, these are search strategies favored by information seekers who are already expert in the domain (Bates, 1979; Stoan, 1984; Ellis, 1989; Drabenstott, 2003). Other reasons for the importance of investigating citation uses in course materials come from 1) the increasing use of constructivist approaches to instruction, 2) recent increases in online learning, and 3) initiatives to encourage interdisciplinary and multidisciplinary education. Citations and web links in online learning materials can make the cited materials immediately accessible to the learner -reducing the effort to retrieve the material and removing one barrier to information access. Careful organization and presentation can help new learners construct knowledge and make meaning by increasing their awareness of the connections inherent in related literature on Instruments of Cognition 5 a particular concept, theory, method or procedure. Understanding current citation practices and uses is also a prelude to developing better and automatic tools that enable the authoring and use of citations and web links as cognitive instruments. Related Literature Students’ use of citations has been studied in at least two different areas: Human information behaviors and discourse analytics. Human information behaviors research is wide-ranging; it includes information seeking, searching, use, and users. A significant body of literature exists in this area and Case (2002) provides a good review of the field. An early and one of the largest catalog use studies done in America reported “instructors should be warned against exclusive oral citation of bibliographical references since it often results, on the part of the students, in misunderstanding and misstating of the authors’ names.” (Jackson, 1958, p. 2). More recently, OCLC reported on the information habits of college students in the U.S. They found that college students predominantly use the Web search engines to find information for assignments, select web resources themselves except when directed otherwise by the teachers or teaching assistants, prefer remote access, use print sources least but like to have print copies of even electronic resources, and have difficulty finding complete articles (with charts, tables, and photographs), critical analyses, bibliographies, locations of books and journals, and descriptive analyses (OCLC, 2002). These findings make clear that students are engaged in learning even as they complete assignments, need written citations, and appear to need guidance in selecting information resources and understanding subject relationships at all stages of the research process. Learning is often studied using either a behavioral or a cognitive approach. Two Instruments of Cognition 6 other approaches to studying learning include the psychoanalytic and humanistic views. The cognitive approach to learning focuses on the mental processes of knowing, while the behaviorists’ approach emphasize observable behavior, especially in terms of responses to stimuli or conditions. The psychoanalytic approach investigates the unconscious processes of learning; the humanistic studies the making of meaning as the core of learning. Constructivist learning theory often blends important concepts from all four approaches: social and cultural aspects of learning including affect or emotion, conation, and meta-cognition or reflection about learning (Steffe & Gale, 1995). In the last decade, constructivism has also emerged as a philosophical underpinning for online (virtual) learning environments (Alessi & Trollip, 2001). Constructivists creating online materials try to facilitate rich interaction by including elements such as multimedia, hypermedia, interactivity, and cooperative learning (Jonassen & Grabinger, 1990). The learner’s manipulations of representations and interactions in the online environment are considered to add a positive dimension to knowledge construction although this view has been challenged and debated (McKendree et al, 1995; Alessi & Trollip, 2001). Nevertheless, there have been few studies that look at the role of citations and web links in this environment. Citation refers to both cited work and the act of citing (Frost, 1978). Web links are hyperlinks, which may be web citations; that is, a bibliographic citation may be presented as a hyperlink, which accesses the cited work when clicked. Citations and web links are important phenomena in the constructivist, interdisciplinary online learning context because 1) citations are concept symbols (Small, 1978), 2) as hypertexthypermedia based technology, web links are information pathways to a wide landscape of Instruments of Cognition 7 information available at the click point to the learner (McKnight et al, 1993; Jonassen, 1993), 3) learning exists within specific discipline discourse communities (Bizzell, 1982, Bazerman, 1981, Myers, 1985) which the links enable or make evident (Tosca, 2000), and 4) successful interdisciplinary learning starts with modest ambitions which may be made visible through citation networks that show the intellectual spaces of a topic or problem (Klein, 1996; Chen, 1999). Instruments of cognition Many studies have devised classifications or taxonomies for categorizing citations (Lipetz, 1965; Chubin & Moitra, 1975; Moravcsik & Murugesan, 1975, 1978; Frost, 1978; Smith, 1980; Duncan et al, 1981; Duncan & McAleese, 1982; Nanba et al., 2000; Thelwall, 2003). Reviews are available (Borgman & Furner, 2002, Liu, 1993; Small, 1982; and Weinstock, 1971) as well as discussions about web-based citation analyses (Cronin, 2001; Almind & Ingwersen, 1997). Only a few, highly pertinent ones, are discussed here. Based on user perceptions and needs in the learning context and those of citing authors, Duncan et al proposed the qualification of citations with two types of relationships, descriptive (which describes the type of resource such as Book, Conference Paper or Review Article) and relational (which describe the type of citation as used by the referring author). The relational qualifiers range from paying homage, background reading, and bibliographical leads to statistics, disputing, and current concerns. Citation type and instructional goals can be correlated. For example, a glossary can be created using “definition” citations; examples can be included using “further detail” or “illustration” citations. The complete list of relational qualifiers is shown in Table 1. [INSERT Table 1 HERE] Instruments of Cognition 8 Such a qualified citation indexing system is discussed in Duncan & McAleese (1982). Frost (1978) investigated three citation types in the humanities based on nature (form) of the information resource cited: 1) primary sources, 2) secondary sources, and 3) either primary or secondary. Acknowledging pioneering work and intellectual indebtedness (trademarks of scholarly bricklaying in the sciences) were seldom used and the practice of building upon previous research was “conspicuously lacking” in the sample of the literature investigated. Moravcsik & Murugesan (1975), Chubin & Moitra (1975) and Murugesan & Moravcsik (1978) developed typologies to analyze citation quality and function by classifying citation nature and context (the citing areas). Their research objective was to understand patterns in citing behaviors of authors and identify variations that may exist between specialties and countries. Findings confirmed that citing practices vary in form, content (whether reported individually or as co-authors from a lab) and outlet (the country and journal of publication) of the citing articles. Small (1978) claimed that in research fronts cited documents tend to be associated with the same words/phrases each time they are cited. The citations came to stand for these concepts, “standard symbols” through processes of consensual agreement. Cozzens (1982) successfully showed that such a document could have a split citation identity if two different networks of researchers applied different aspects of it and hence the citation could have two symbolic roles assigned to it. These studies have potential implications for the use of citations as learning aids in knowledge construction. Two final studies are by Kim (2000) and Thelwall (2003). Kim examined the motivations for hyperlinking in scholarly articles and identified three categories of Instruments of Cognition 9 reasons: scholarly, social and technological. In the scholarly category, the provision of additional or background information was the most common reason, with example or illustration as a second. Thelwall suggested that creation motivations for web links on academic websites and citations in journals were likely to be different because they are very different phenomena. Based on a study of 100 inter-site links, four new types of motivations for web links and citations were postulated: ownership for links acknowledging authorship or co-authorship of a resource, social for links with primarily a social reinforcement role, general navigational for those with a general navigation function, and gratuitous for those that serve no communication function at all. He found that instead of being primarily socio-cognitive, no links were cognitive and the gratuitous were not even social. In summary, a number of variables influence citation behaviors in the course of research: the citation, the form of the cited work, the discipline in which the citing is being done, the motivation of the citer, the citation context, and the symbolic referent role. Geographic Information Science There is little documentation about citing behavior for instructional purposes and none was found about citation networks in Geographic Information Science although literature from geo-visualization may be potentially relevant (Skupin, 2001; Skupin & Fabrikant, 2003). The abbreviation GIS is used for both Geographic Information System and Geographic Information Science. Geographic Information Systems were initially custom-designed and developed as teaching tools, but these have largely been abandoned now and the field has standardized around two commercial products, IDRISI and ESRI’s Instruments of Cognition 10 ArcGIS. In 1991, the University Consortium for Geographic Information Science (UCGIS), was formed. This is an organization devoted to the development of teaching GIS as an academic discipline includes photogrammetry, remote sensing, geodesy, cartography, surveying, geography, computer science, cognitive science, spatial statistics, and other disciplines with interest in spatial data. UCGIS has increasingly reflected the constructivist view, shifting from a focus on teaching to a focus on learning and constructing knowledge (UCGIS, 1997a; UCGIS, 1997b). Learning GIS is a challenge; it is about solving problems using the basic concepts, theories and tools that come from multiple subjects (Nyerges & Chrisman, 1989; Nyerges, 1993). Methods Used To investigate student use of citations and web links embedded in online teaching and learning materials, we asked two research questions: 1) What are the different types of citations and web links that can be found in instructional materials? 2) Why do students use citations and web links? Data collection and analysis methods included: 1. Citations and web links: We counted the number of citations and web links in the course materials of an undergraduate senior/graduate level course taught in Spring 2003 at a major southwestern university. 2. Citation and link content/context analysis: We examined the citations and links and categorized them using a number of typologies. 3. User study: We designed instruments comprising written surveys and follow-up interviews. Instruments of Cognition 11 4. Usage tracking: We developed computer software to follow student activity on web links in course pages. The Course: The choice of course and discipline derived from a larger study that is trying to map the intellectual structure of Geographic Information Science (Coleman, 2003). For the study reported here, investigating the phenomenon of student use of citations and web links, we analyzed materials from a graduate course that is also offered at the senior undergraduate level. The course is based in the Geography & Regional Development Department, a unit of the College of Social Sciences, and the syllabus describes geography thus: “Geography is not merely the description of where things are, but is a rich body of ideas about spatial relationships – why things are where they are, and how they function there.” Geographic information science is defined as “the study of the complex interactions between the physical landscape and human beings, and its representation in map form.” The course is designed with three components: Lectures (in-class), Labs (physical laboratories with ArcView and Arc/INFO), and Interactive Learning Modules (materials in an online course management system). The course designer and instructor claim the use of constructivist principles in the design of the course. The Interactive Learning Modules (ILMs) provide the theory (conceptual knowledge) to support the practical labs. The lectures serve as the bridge behind the theoretical modules and the practical labs. Our analysis of citations and web links is limited to the ILMs computer-based lectures that the students completed on their own time. All the ILMs had a common structure. Each ILM had a learning outcome; Instruments of Cognition 12 definitions and explanations presenting concepts; and visualization and illustration objects such as maps, images, cartoons, pictures, datasets, and interactive services. Quizzes tested understanding of the concepts presented and were the final component of the first two ILMs. The ILMs existed in WebCT, a web-based online course management system, available 24/7. The course lasted the traditional 16 weeks of the semester. There were ten ILMs, a glossary, 12 lectures and 12 labs. The course had a pre-requisite; there was no text. The Participants: Forty-five students (of a class enrollment of 90) agreed to participate in the study; the actual number of student participants who participated fully turned out to be smaller (mean n=26). Data collection: Three main methods were used to collect data. First, the participants completed online web surveys. Two types of online survey instruments were developed. The first was a demographic survey to gather such information as gender, age, ethnicity, major area of study, and educational level. The second online survey was administered only to the students participating in the study; it focused on their usage of citation and web links. Students completed ten such surveys, each of which was hyperlinked from the ten ILMs and was thus ILM specific. Students who were not a part of this study were not shown the surveys. Second, we conducted focus group interviews with the students at the end of the semester, when a majority of the modules had been completed. These interviews are used to verify data gathered in the written surveys and fill in gaps about student uses and Instruments of Cognition 13 perceptions of citations and links. Third, we planned to track the students’ access patterns using custom transaction logging scripts; because of methodological complexities this is not included here. Results and Findings Rates of Participation Student participation in completing the online surveys and attending the focus group interviews varied, as would be expected with this complex protocol. Therefore, the findings should be interpreted with caution. The mean rate of participation is 26 students per module and the mode is 25. Table 2 provides the number of students who completed the survey for each module. Twenty-eight students participated in focus group interviews conducted over three days. [INSERT TABLE 2 HERE] Participant Profiles Table 3 shows the demographic characteristics of the student participants in terms of their gender, ethnicity, age, marital status, level of education, major area of study, number of hours of work in a job, and prior education. The typical student in this GIS course was a male, Caucasian, undergraduate senior in the age group of 20-30. He was not from the state where the university is located, was majoring in Regional Development, and worked fewer than 20 hours a week in a job. [INSERT TABLE 3 HERE] Nature of Citations and Links Links for course administration and internal navigation were not included in this analysis. Instruments of Cognition 14 The ILMs (ten citing documents) had a total of 16 bibliographic citations and 20 web links. All of the ILMs except the last one (#10) had a glossary. Definitions given in the narrative of the ILMs were rarely duplicated in the glossary. The total number of terms in the glossary was 315. The total approximate size of the ILMs was 50.2 MB. The references (cited documents) are listed in Appendix 1. Using the Duncan classification shown in Table 1, we analyzed the form (genre) and context of the citations and web links. The Duncan classification was chosen because the “qualified index” was developed using two educational technologists (Duncan, et al, 1981, p. 73) and tested with users (Duncan & McAleese, 1982, p. 81). Additionally, their own paper references are categorized with relational qualifiers, thereby serving as a model (Duncan et al, 1981, p. 77-79; Duncan & McAleese, 1982, p. 84-85). Context (purpose) was inferred by looking at the area around the citation. Thus, looking at row 1 in Table 4 (the serial number indicates the ILM number), ILM 1 had five citations, all of which were books, and the context of the citations reveal definition and illustration as the citing purpose. GIS, not unusually, is a visually rich discipline and there were a total of 130 images in the ten ILMs. No citation was used to criticize or dispute. Instead the predominant contexts of the citations are definition, explanation, examples, and illustration. Examples are not in the Duncan classification and we added them. To determine form and content, we used the LCSH Free Floating Form Subdivisions (LCSH, 2000). The predominant form of the citation is books. The predominant text content is quotation followed by illustrations (images). Table 4 shows the breakdown. The number of unique bibliographic citations is only five. [INSERT TABLE 4 HERE] Instruments of Cognition 15 Moving to the analysis of links, we again used the Duncan categories to characterize context. The results of our analysis are presented in Table 5. The LCSH Free Floating Subdivisions does not include web sites which would have been the most generic description of the links we observed, but we have used this term qualified by the top domain as most appropriate initially. The column “Label” in Table 5 refers to how the hyperlinks were presented, using the following typology: Text label: any brief label; URL label: any label containing an url; Directive label: any label giving directions such as for further detail; Graphic label: any graphic/iconic hyperlink. Websites are the predominant form of the web links; links appeared to have been provided for further detail and links to .gov and .edu sites predominate. [INSERT TABLE 5 HERE] Citation and link use behaviors by students Rather than aggregating responses, we analyzed the results by module because participation in the surveys for each module varied considerably. The number of student uses and non-uses of citations and links is shown for each module in Table 6. The number of students who did not use the citations and links is greater than those who used them. [INSERT TABLE 6 HERE] Reasons for Use and Non-Use Students participating in the online surveys gave many reasons for not reading the material cited or linked. These include: Instruments of Cognition 16 Not enough time Too much information; too much to absorb; too much material to read Not interesting; Uninteresting I already know this information; I am familiar with these websites from work Not in a position to comment now Most information in the ILMs Not relevant to labs (GIS applications) Technical problems such as access restrictions and the disappearance of one link which changed address during the course period (link in ILM #4, GeoStat, University of Virginia Library). Also given as a reason for not following links: o Unable to access link or unable to log in o Unable to locate citation in library Students also reported a variety of reasons for using citations and links and these are: Starting points: “Citation offers a starting point from which to become really familiar with the history and information about GIS.” Further details: “I wanted to receive more information about what ArcView has to offer.” Clarification: “I was curious to know more about Ian McHarg. Confused because I thought he did something else.” Cognitive Instruments We approached this study by considering citations and links as instruments of cognition. Written comments on the survey were analyzed in terms of five rudiments of cognition that constructivism considers in instructional design and delivery: anxiety, Instruments of Cognition 17 arousal, attention, motivation, and self-regulation (there are a number of other cognitive elements, for example, memory, meta-cognition, and schemas, that are not examined). Additionally, the focus group interviews probed the reasons for use, non-use, and quality of the citations and links. Anxiety: Anxiety is often considered an inhibiting factor in learning and has been shown to impair a wide variety of cognitive functions including attention, concept formation, memory, and problem solving. Tests and exams have been shown to be precursors of anxiety in educational situations. Citations did not generate any anxiety. A majority of the students felt that they could go back and find these any time they wanted. A few recorded the citation and explained that they probably would not follow it until it was needed for a work or class project. Links, on the other hand, generated a greater negative or apathetic response: the student “did not want to get distracted,” “was not interested,” or “did not have time.” Only a few were “curious” and “desired” to see more. Two comments from the group that did not follow the links are revealing: 1) “These seemed to be just more information. I am currently only interested in learning the basics, thus expanding to other websites would not be helpful.” 2) “I was in a sort of rush to get through most of the reading. I just never really thought about looking at the links. They sort of just passed my eye.” This is not anxiety; nevertheless, we probed the non-use further in the focus groups and received some interesting answers which are discussed further below. Arousal: The Yerkes-Dodson law (1908) shows an inverted U-shaped function between arousal and performance. This means that a moderate level of arousal results in the maximum performance on a task. Both high and low levels of stimulation result in Instruments of Cognition 18 minimum performance. There is also a relationship between arousal and curiosity. There is an optimal arousal level for individuals and when this level drops it can increase the learner’s pursuit of exploratory behavior. According to Berlyne (1960), curiosity was a state of “conceptual conflict” brought about by doubt, perplexity, ignorance, contradiction, or irrelevance. Do citations and web links play a role in arousal? One student wrote: “I just clicked the link and simply followed the material, just glancing around for my own benefit and interests.” But, another one wrote: “Tired of reading; did not want to read anymore; it did not seem important.” It would be simplistic, and untrue, to conclude that citations and links do not arouse much enthusiasm and ignore the students who did indicate they were “curious.” Much more research is needed to confirm this, especially since, as we shall see below, the presentation of the citation or link appears to be a factor influencing use. Arousal is also related to motivation as curiosity is a feature of both, and this relationship needs to be probed further. Attention: Gaining the attention of students has been considered the first step in successful instruction according to Gagne’s Conditions of Learning (Gagne, 1987). How well did the citations and web links gain the attention of the students? We found that citations, generally, did not gain the attention of students; when they did (as in the case of the student who was “reminded of the book from 416A”), the citation appears to have triggered recognition and/or recall. Some web links were successful in gaining student attention. Here is what students said: “ I loved the graphics; wish there were more of them”; “The interactive pop-ups got my attention” ; “The maps and data opened my eyes to a new world.” Motivation: Cognitive theories tend to deal with intrinsic motivation such as goals, and Instruments of Cognition 19 behavioral theories of learning tend to deal with extrinsic rewards. Malone’s theory of intrinsic motivation (1981) discusses the importance of challenge, control, fantasy, and curiosity in intrinsically motivating learning environments. In the case of a few students, citations and web links did stimulate curiosity. One student “wanted to know about further research in DBMS” and used web search engines to find more information. However, in the case of several others, citations and links presented nothing new. “I use TIGER, DLGS, and DEMs in my work” commented a non-user. Self-regulation: Self-regulated learning environments have three characteristics: they are intrinsically motivating because participating in the activity has its own rewards, learners plan and monitor their own learning, and learners control the learning to suit their learning styles (Zimmerman, 2000). The structuring of the course with lectures, labs and online, interactive learning materials was meant to create a partially self-regulated learning environment just as the citations and web links were meant to encourage exploratory learning behaviors. Students liked the 24/7 availability of the modules, but they did not like the fact that feedback was not immediate; that the links were neither rated nor typed, i.e., identified in terms of their importance for learning the material as well as in terms of information quality; and that they failed to clearly identify what would be found when one arrived at the other end. The students who followed the links were in the minority. But, equally clearly, they were pursuing their interests and were aware that they were constructing their own map of how GIS would be helpful to them in many different ways: 1) “ I looked up through the human geography because that is what I am interested in.” ; 2) “Read the intros and signed up for AAG membership” ; 3) “very good and very exciting; pivotal to my understanding of this ILM” ; 4) “ best module and the Instruments of Cognition 20 citation reminded me that the book in 416a is relevant; becoming increasingly difficult, but looking through the datasets opened new worlds.” Several suggestions emerged from the focus group interviews that offer solutions to the question of how citations and web links can better serve as instruments of cognition for learning. 1. Categorize the links and citations in course materials in lists of Required, Recommended, and Optional. Required List: This list should order and structure citations and web links by articulating clearly the role (function or purpose of citing/inclusion in list). Additionally, required links and citations should be accessible at clickpoint, i.e., contain what must be followed/read. Students also emphasized that these cited works must present the essential foundational concepts for learning. Foundational purposes (which can be given as relational qualifiers) include providing definitions, descriptions, illustrations, examples, criticism, methodology, and experimental detail. Recommended list: These links and citations are the works recommended for further following and reading. They must also include the reason for the recommendation, such as background reading, another point of view, etc. Optional list: Links and citations could range widely; bibliographies and webliographies compiled by others, and all sorts of other interesting, additional, materials should go here. 2. Present web links as web citations. In other words, don’t just hyperlink, organize them as a web citation by providing full bibliographic information. Instruments of Cognition 21 3. Highlight citations and web links distinctively. The small size of the citations at the bottom of the screen/page and the same convention (blue hyperlinks) used for most links (navigational and informational) resulted in students often ignoring them because they appeared to be invisible. 4. Compile and make available online class reading lists, annotated bibliographies, summaries, and lists of examples. 5. Integrate the GIS computing environments into the web learning environment (WebCT). By this students meant that it would have been helpful to link directly to or between the GIS software and the ILMs. Students found “living” in three different class environments confusing and time-consuming. The three perceived environments are: real-life interaction in class lecture and lab, the WebCT ILM (GUI) world, and the ArcINFO (command-mode) world. Desiring seamless integration of theory and practice they often ignored the ILMs and lectures and became focused on the GIS Labs. The GIS Lab was perceived as the real practice environment that helped them learn new skills and knowledge; conceptual knowledge, they felt, could be “retrieved’” and they did not feel the need to carry it around in their heads. Thus, there were many requests for citations and links to components (snippets) of information in the ArcView and ArcINFO Labs (Help and Tutorials). Hyperlinks to illustrations (for example about projections), procedures, commands (ArcINFO commands), experimental details, and applications were sought. 6. Rate the quality of the cited work. Students were very interested in having an expert’s judgment of materials; instructors should explain why and what in a Instruments of Cognition 22 particular book or article is good for learning. 7. A rare student indicated her awareness of citation searching: “I will use this citation to search for related information; I can do this by Google or in library.” Yet another student showed her awareness of scholarly knowledge. “It would be nice to see criticisms of methods or concepts rather than just definitions and explanations.” Summarizing, not many students used citations and links, but those who did reveal that citations are cognitive instruments and that both links and citations must be organized and presented better for their full potential in learning to be realized. Hypermedia/hypertext literature shows that findings about the need for link labeling and typing are not new. Quentin-Baxter (1999) identified the need for classification of the hypertext information presented to help improve learning. Blustein (2000), in evaluating the performance and preference of readers of scientific articles, found that structural and definitional links were preferred to semantic ones. Link highlighting improved browsing efficiency and many ways of highlighting graphical and textual links are presented in Olston & Chi (2003). Additionally, research from areas such as hypertext criticism (Morgan, 2002); adaptive hypermedia, i.e., learner modeling; (Brusilovsky & Maybury, 2002); concept mapping (Novak, 1998); and external representations (Rogers & Scaife, 1996; Scaife & Rogers, 1998) suggest diverse organizations and displays of links for improving cognition, among other things such as collaboration, communication and performance in learning tasks. Conclusion Citations and links are like the windows and doors to a knowledge storehouse. Teachers, experts in the area of study, expend a fair amount of thought and effort on the Instruments of Cognition 23 selection of citations and web links they include in course materials. At the senior undergraduate level, one may reasonably expect students to use these citations and links. Such use could be varied: reading, using them in assignments, discussion with peers, reflecting on them, abstracting ideas from them, etc. However, information and cognitive overload is a very real problem that students deal with in learning an interdisciplinary area like GIS. Many students found even the small number of citations and links in the course cognitively “distracting,” “interesting,” “burdensome,” and “difficult to use.” Additionally, the online environment continues to be a challenge for students, thereby contributing to the cognitive overload. Advances in information visualization, autonomous citation indexing, and adaptive hypermedia may offer solutions for presenting citations and links in more user-friendly and usable ways. Until then, current practices in citation and web links for online learning should facilitate clearer organization and presentation. As knowledge structures, citations and links can be organized for cognition by indicating relationships such as their role or function, form, context, and importance for learning. More research exploring the relationship between citations, links, and cognition is also needed. An important question for such future experimental research is: Will networks of citations and web links in course materials augment or hinder interdisciplinary learning?